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The First Method for Finding $\beta_0$ and $\beta_1$ 


8.5.2 The First Method for Finding /? 0 and fi 1 

Here, we assume that x -'s are observed values of a random variable X. Therefore, we can summarize 
our model as 


Y=p Q +p x x+e, 

where e is a MO, a 2 ) random variable independent ofX First, we take expectation from both sides to 
obtain 


EY = p 0 + p l EX+E[c\ 

= Pq+P\EX 


Thus, 


Pq = EY-p x EX. 


Next, we look at Cov(X, Y), 

Co\'(X, Y) = Cov(X, fi () + [i\X + c) 

= /? 0 Cov(X, 1) + p x Cov(X, X) + Cov(X, c) 

= 0 + P x Cov(X, X) + 0 (since X and r are independent) 
= /?lVar(X). 


Therefore, we obtain 


Pi 


Cov(X, Y) 
Var(X) ’ 


Po = EY—P x EX. 


Co v(X,Y) 

Now, we can find fi 0 and /? 1 if we know EX, EY, Var( V) . Here, we have the observed pairs (jc 1 ,> , 1 ) 
, (x 2 ,t 2 )> so we ma y estimate these quantities. More specifically, we define 
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X1+X2 + ... + x n 

X = -, 

n 

y x +y 2 + --- + y n 
y =-> 

n 

n - 

S XX = Z (x r x) 2 , 

i= 1 

n - - 

s xy= Z (*/“*XT/“T)- 

i= 1 

We can then estimate and fi x as 

A 

h 

A 

h 

The above formulas give us the regression line 

A A 

y = P, o+Ai*- 

For eachx., the fitted value y. is obtained by 

A A 

T/ = P() + P\ X T 

Here, j> • is the predicted value ofj^- using the regression formula. The errors in this prediction are 
given by 

e i = yrh 

which are called the residuals. 

Simple Linear Re g ression 

Given the observations (x x ,y x ), (x 2 ,y 2 ), "S (x„,y„), we can write the regression line as 

y = Po + fi x x- 

We can estimate /? 0 and fi x as 
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A 


s 


xy 


A - A - 


Po = y~P\ X > 


where 


n - 

s xx = Z ( x i - -A 2 , 

i= 1 

n - - 

s xy= Z iXi-xfoi-y). 
i= 1 

For eachx ; -, the fitted value y i is obtained by 


The quantities 


A A 

yi = Po + Pi x v 



are called the residuals. 


Example 8.31 

Consider the following observed values of (x t ,y ■): 

(1,3) (2,4) (3,8) (4,9) 

1. Find the estimated regression line 


A A 

y = Po + P\ x , 


based on the observed data. 

2. For each*-, compute the fitted value ofy ■ using 


A A 

h = A) + P\ x v 

3. Compute the residuals, e i = y i -y • and note that 
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I^o. 

i=l 


• Solution 

o 1. We have 




3 + 4 + 8 + 9 


s xx = (1 - 2.5) 2 + (2 - 2.5) 2 + (3 - 2.5) 2 + (4 - 2.5) 2 = 5, 
s X y = (1 - 2.5)(3 - 6) + (2 - 2.5)(4 - 6) + (3 - 2.5)(8 - 6) + (4 - 2.5)(9 - 6) = 11. 

Therefore, we obtain 


fil 

A 

h 


s xy 11 

— = — = 2.2 
Sw 5 


6 - (2.2)(2.5) = 0.5 


2. The fitted values are given by 


y t = 0.5 + 2.2x ■, 


so we obtain 


Pi = 2.7, j> 2 = 4.9, j> 3 = 7.1, j> 4 = 9.3 

3. We have 

<?1 =y x -j>! = 3 -2.7 = 0.3, 

e 2 = T2“i ) 2 = 4 “ 4 - 9= _0 - 9 > 

e 3 = - J> 3 = 8 - 7.1 = 0.9, 

^4 = T4 - J>4 = 9 - 9.3 = - 0.3 

So, + e~, + e 3 + — 0. 


We can use MATLAB or other software packages to do regression analysis. For example, the 
following MATLAB code can be used to obtain the estimated regression line in Example 8,31 . 
x=[l;2;3;4]; 
x0=ones(size(x)); 
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y=[3;4;8;9]; 

beta = regress(y,[xO,x]); 

Coefficient of Determination (^-Squared): 

Let's look again at the above model for regression. We wrote 

Y= Pq+P x X+£, 


where e is a MO, a ) random variable independent of X. Note that, here, X is the only variable that 
we observe, so we estimate 7 using X. That is, we can write 


y = P q +P x x. 


The error in our estimate is 


Y~Y=e. 

Note that the randomness in Y comes from two sources: X and c. More specifically, if we look at 
Var(7), we can write 

2 

Var(7) = /ij Var(V) + Var(r) (since X and c are assumed to be independent). 

The above equation can be interpreted as follows. The total variation in Y can be divided into two 

2 

parts. The first part, p\ Var(X), is due to variation in X. The second part, Var(r), is the variance of 

error. In other words, Var(e) is the variance left in Y after we know X. If the variance of error, Var(r), 

is small, then Y is close to 7, so our regression model will be successful in estimating Y. From the 
above discussion, we can define 

/iTVar(V) 

2 = _ 

' Var (Y) 

as the portion of variance of Y that is explained by variation inX From the above discussion, we can 
also conclude that 0 < p < 1. More specifically, if p “ is close to 1, Y can be estimated very well as a 

linear function ofX. On the other hand if p 2 is small, then the variance of error is large and Y cannot 

Cov(X,F) 

be accurately estimated as a linear function ofX. Since /?, = Var ^ , we can write 

/IjVarfX) [Cov(X, Y)\ 2 

p- = - = - ( 8 . 6 ) 

Var (7) Var(X)Var (7) 

The above equation should look familiar to you. Here, p is the correlation coefficient that we have 
seen before. Here, we are basically saying that if Xand Y are highly correlated (i.e., p(X, Y) is large), 

then Y can be well approximated by a linear function of X, i.e., 7 ~ 7 = /? 0 + ft x X. 
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We conclude that p 2 is an indicator showing the strength of our regression model in estimating 
(predicting) Y from X. In practice, we often do not have p but we have the observed pairs (x } , y f ), 

(x 2 ,y 2 ), We can estimate p 2 from the observed data. We show it by r 2 and call it R- 

squared or coefficient of determination. 

Coefficient of Determination 


For the observed data pairs, (x 2 ,y 2 ), •••, (x , v ), we define coefficient of determination, r 2 

as 



s xx s yy 


where 


n - n - n — — 

s xx = Z (*t ~ x s yy = Z 0 / “ F) 2 , ^ = Z ( x i _ x )0- j)- 

i =1 /=1 / 1 

We have 0 < r 2 < 1. Larger values of r 2 generally suggest that our linear model 


AAA 

w = Ai + fe 


is a good fit for the data. 

Two sets of data pairs are shown in Figure 8.12. In both data sets, the values of the y ( 's (the heights 
of the data points) have considerable variation. The data points shown in (a) are very close to the 
regression line. Therefore, most of the variation iny is explained by the regression formula. That is, 

A 

here, they Is are relatively close to they Is, so r 2 is close to 1. On the other hand, for the data shown 

in (b), a lot of variation iny is left unexplained by the regression model. Therefore, r for this data 
set is much smaller than r 2 for the data set in (a). 



Figure 8.12 - The data in (a) results in a high value of r 2 , while the data shown in (b) 

results in a low value of r . 
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Example 8.32 

For the data in Example 8.31 . find the coefficient of determination. 
• Solution 

o In Example Example 8.31 . we found 


We also have 


5 vy = (3 - 6) 2 + (4 - 6) 2 + (8 - 6) 2 + (9 - 6) 2 = 26. 

We conclude 



5 x 26 


0.93 


<— previous 
next —» 
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